Decoding Word Embeddings with Brain-Based Semantic Features

Word embeddings are vectorial semantic representations built with either counting or predicting techniques aimed at capturing shades of meaning from word co-occurrences. Since their introduction, these have been criticized for lacking interpretable dimensions. This property limits our understanding the features they actually encode. Moreover, it contributes to “black box” nature tasks in which used, since reasons embedding performance often remain opaque humans. In this contribution, we explore properties encoded by mapping them onto vectors, consisting explicit and neurobiologically motivated (Binder et al. 2016). Our exploration takes into account different types embeddings, including factorized count vectors predict models (Skip-Gram, GloVe, etc.), as well most recent contextualized (i.e., ELMo BERT). analysis, first evaluate quality a retrieval task, then shed light on that better each type. A large number probing is finally set assess how original mapped perform discriminating categories. For identify relevant show there correlation between encode those features. study sets itself step forward aspects captured vector spaces, proposing new simple method carve human-interpretable distributional vectors.